Motion compensated interpolation

ABSTRACT

This invention provides a way of performing improved motion compensated interpolation of moving images, such as television, using motion vectors of variable reliability. By taking into account the reliability of the motion vectors, produced by a separate motion estimation device, a subjectively pleasing interpolation can be produced. This is in contrast to simple motion compensated interpolation, taking no account of motion vector reliability, which is often degraded by objectionable switching artifacts due to unreliable motion vectors. The invention can be used, for example, to improve the performance of motion compensated standards converters used for converting between television standards with different picture rates. The invention allows a gradual transition between motion compensated and non-motion compensated interpolation depending on the reliability of the motion vector used. This is achieved by modifying the temporal interpolation timing, using a look up table, controlled by a vector reliability signal produced by the motion estimator. Effectively this adapts the motion trajectory of the interpolated output pictures.

BACKGROUND OF THE INVENTION Field of the Invention

The invention relates to a method and apparatus for processing film orvideo signals which avoids objectionable switching artifacts whenperforming motion compensated temporal interpolation. This is useful,for example, in the inter-conversion of television pictures withdifferent picture rates. The invention is also suitable for methods andsystems which use motion adaption instead of motion compensation.

DESCRIPTION OF THE RELATED ART

In this application, the term picture is used as a generic term coveringpicture, field or frame depending on the context. Film and televisionprovide a sequence of still pictures that create the visual illusion ofmoving images. Providing the pictures are acquired and displayed in anappropriate manner the illusion can be very convincing. J. Drewery, inreference 9, eloquently describes the nature of the illusion. In moderntelevision systems it is often necessary to process picture sequencesfrom film or television cameras. Processing which changes the picturerate reveals the illusory nature of television. A typical example is theconversion between European and American television standards which havepicture rates of 50 and 60 Hz respectively. Conversion between thesestandards requires the interpolation of new pictures intermediate intime between the input pictures. Many texts on signal processingdescribe the interpolation of intermediate samples, for a properlysampled signal, using linear filtering. Unfortunately, linear filteringtechniques applied to television standards conversion may fail to work.Fast moving images can result in judder, blurring or multiple imageswhen television standards are converted using linear filtering. Thisillustrates the illusory nature of television systems. The difficulty ofprocessing television signals is because they are under-sampled in aconventional Nyquist sense. Further details can be found in reference23.

Many people have expounded the benefits of motion compensation as a wayof overcoming the problems of processing moving images (references 2, 3,4, 5, 11, 13, 15, 16, 17, 18, 19, 21). Motion compensation attempts toprocess moving images in the same way as the human visual system. Thehuman visual system is able to move the eyes to track moving objects,thereby keeping their image stationary on the retina. Motioncompensation tries to work in the same way. Corresponding points onmoving objects are treated as stationary which avoids the problems dueto under sampling (reference 3, 25). In order to do this it is assumedthat the image consists of linearly moving rigid objects (sometimesslightly less restrictive assumptions can be made). In order to applymotion compensated processing it is necessary to track the motion of themoving objects in an image. Many techniques are available to estimatethe motion present in image sequences (references 1, 2, 3, 4, 8, 12, 14,20, 24).

With suitable input pictures motion compensation has been demonstratedto give a very worthwhile improvement in the quality of processedpictures. Under favourable conditions the artifacts of standardsconversion using linear filtering, that is judder, blurring and multipleimaging, can be completely eliminated. Motion compensation, however, canonly work when the underlying assumptions are valid. In unfavourablecircumstances the assumption that, for example, the image consists oflinearly moving rigid objects is violated. When this happens the motionestimation system, necessary for motion compensation, is unable toreliably track motion and random motion vectors can be produced. Whenthe motion estimation system fails the processed pictures can containsubjectively objectionable switching artifacts. Such artifacts can besignificantly worse than the linear standards conversion artifacts whichmotion compensation is intended to avoid.

Ideally a motion compensated processing system would provide the fullbenefits of motion compensation on suitable pictures while performing aswell as, or better, then conventional linear processing on unfavourablepictures. In order to achieve this the system must change betweeninterpolation methods depending on the suitability of the pictures formotion compensated processing. The system would, therefore, adaptbetween motion compensated and non-motion compensated processing As withadaptive television systems in general it is inadvisable for there to bea sudden switch between interpolation methods. Such a switch can, ofitself, produce switching artifacts when the pictures are ofapproximately equal suitability for motion compensated or non-motioncompensated processing. A system which gradually changes from motioncompensated to non-motion compensated processing according to thesuitability of the pictures is said to exhibit graceful fall-back. Thenon-motion compensated processing method is known as the fall-back mode.

In order to implement a motion compensated system with gracefulfall-back it is necessary to know when the pictures are unsuitable formotion compensation. This depends on whether the motion estimator canproduce reliable vectors. Hence it is necessary for the motion estimatorto indicate whether the vectors it is producing are reliable. R Thomson,in reference 22, provides an excellent discussion of the above argumentsand describes how, in a phase correlation type motion estimation system,an indication of the reliability of motion vectors is given by therelative height of the correlation peaks produced. Other motionestimation systems can also be designed to provide an indication ofvector reliability. A block matching motion estimator, for example,could provide the match error for the selected vector as a measure ofvector quality.

Another requirement for motion compensation with graceful fall-back is asuitable, non-motion compensated, fall-back mode. One obviouspossibility is to fade between a motion compensated algorithm and aconventional linear filtering algorithm. This approach, however, has anumber of disadvantages. Unless the pictures are particularly suitablefor motion compensation the output pictures would include a smallproportion of a conventional interpolation with its attendant artifacts.The presence of these artifacts, albeit at a low level, might besufficient to undermine the reason (artefact free pictures) forperforming motion compensation in the first place. Nor is linearfiltering particularly suitable as a fall-back algorithm. Linearfiltering only works properly when the picture is stationary or slowlymoving. This is unlikely to be the case when the motion estimator isunable to reliably track motion.

SUMMARY OF THE INVENTION

It is an object of the present invention to allow graceful fallback ofinterpolation systems. This is achieved by gradually changing thetemporal interpolation phase between a full temporal interpolation andselection of the temporally nearest input picture, that is, picturerepeat where the phase of the temporal interpolation is coincident withthe nearest input picture. The degree to which the temporalinterpolation phase is modified depends on the reliability of the motionvector used in the interpolation.

The invention provides a method of interpolating in processing of videoor film signals comprising storing input pixel values of an input signalin an input store, assigning a motion vector to each set of outputcoordinates to be interpolated, providing an indication of thereliability of each motion vector, modifying the temporal coordinate ofeach set of output coordinates depending on the reliability of thecorresponding motion vector, and selecting at least one pixel value fromthe input store depending on the motion vector and the modified temporaloutput coordinate, an interpolated output pixel value is determined fromsaid at least one pixel value. Thus, if the vector reliability isassured, the interpolation phase is coincident with that of the outputpicture phase. As the vector reliability decreases the phase of theinterpolation is shifted towards the temporally nearest input picture.At zero, or a minimum specified, vector reliability the interpolation isequivalent to picture repeat.

The input store may store a plurality of pictures, at least one pixelvalue being selected from each picture, and a corresponding filtercoefficient is selected from a coefficient store for each pixel value,the value of the output pixel is determined from a weighted sum of theplurality of pixel values multiplied by their correspondingcoefficients. The filter coefficients being stored in a second memory.

The temporal coordinates may be modified by using a lookup table, thetransfer characteristic of which is determined by the reliability of themotion vector.

In one aspect of the invention, the motion vector assigned to each setof output coordinates is zero, and a motion detector is used to give anindication of how reliable a zero motion vector is for each set ofoutput coordinates.

The invention also provides a video or film signal processinginterpolation apparatus comprising an input store for storing pixelvalues of an input signal, means providing an indication of thereliability of a motion vector assigned to each set of outputcoordinates to be interpolated, means modifying the temporal coordinateof each set of output coordinates depending on the reliability of thecorresponding motion vector, and a vector processor which selects atleast one pixel value from the input store depending on the modifiedtemporal output coordinate and the corresponding motion vector.

The apparatus may comprise a plurality of multipliers and associatedcoefficient stores, and an adder, the input store being adapted to storea plurality of pictures. The vector processor selects at least one pixelvalue from each picture and a corresponding coefficient for each pixelvalue, the output pixel value being calculated from a sum of theselected pixel values weighted by the corresponding coefficient.

The apparatus may further comprise a motion detector, the motion vectorassigned to each set of output coordinates is zero and the motiondetector supplies an indication of the reliability of a zero motionvector to the means for modifying the temporal coordinate of each set ofoutput coordinates. Alternatively, the apparatus may comprise a motionestimation device which assigns motion vectors to each output coordinateand is adapted to provide an indication of the reliability of eachmotion vector to the means for modifying the temporal coordinate of eachset of output coordinates. The motion estimator may be of the blockmatching type, the indication of reliability is given by the matcherror.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be described in more detail and by way of exampleonly with reference to the accompanying drawings, in which:

FIG. 1 shows the motion trajectory of a linearly moving object;

FIG. 2 shows the motion trajectory for picture repeat interpolation;

FIG. 3 shows a motion trajectory intermediate between those shown inFIGS. 1 and 2;

FIG. 4 shows the relative timing of input and output pictures forconversion between signals of 50 and 60 Hz;

FIG. 5 is a schematic of a standard motion compensated interpolationsystem;

FIG. 6 is a schematic showing an interpolation device according to thepresent invention;

FIG. 7 shows possible transfer characteristics suitable for adaptivemotion compensated interpolation, and

FIG. 8 is a schematic showing an interpolation device according tosecond embodiment of the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The basic assumption underlying motion compensation is that the imagecomprises a collection of linearly moving rigid objects. In motioncompensated processing image processing operations are performed in theframe of reference of the moving object rather than the frame ofreference of the image. This avoids processing problems associated withtemporal aliasing due to under sampling the pictures in time. Motioncompensation and the reasons for it are described in detail in manyreferences for example 3, 11, and 25. Provided the assumption of linearmotion is obeyed then the spatio-temporal trajectory of the objects canbe represented by straight lines in space/time as illustrated in FIG. 1.

Despite the demonstrable success of motion compensated processing someimages (or parts of images) do not conform to the underlyingassumptions. Violation of these assumptions will occur for partiallytransparent or translucent objects (e.g. smoke), changes in shape orlighting and cuts between different scenes etc. Such violations occuroften in typical moving pictures and therefore must be processedacceptably. Small deviations from the assumptions are acceptablyprocessed using motion compensation. As the deviations become largermotion compensated processing becomes less and less acceptable as itbecomes increasingly difficult to find a representative motion vector.For large violations of the motion compensation assumptions the motionestimator will fail completely producing essentially random motionvectors. Nevertheless it is still necessary to produce processed imageseven when the motion estimator has failed completely. In thesecircumstances perhaps the only reasonable interpolatian method is tomake the output picture the same as the (temporally) nearest inputpicture. This is known as picture (or field) repeat in television termsand zeroth order interpolation in signal processing parlance. A motiontrajectory for picture repeat is shown in FIG. 2.

With motion vectors of intermediate reliability an interpolation methodis required between the two extremes of full motion compensation andpicture repeat illustrated in FIGS. 1 and 2. One way to do this is toassume a motion trajectory between those for the two extremes. This isthe basis of this invention. FIG. 3 illustrates such an intermediatemotion trajectory.

To achieve an intermediate motion trajectory the time to which aninterpolated output picture corresponds is modified depending on thetemporal interpolation phase and the motion vector reliability. Thetemporal interpolation phase is the time in the input sequence at whichan output picture is required. The temporal interpolation phase is mostconveniently expressed in terms of input picture periods. For example,consider converting between television signals with 50 and 60pictures/second. The first output picture (at 60 Hz) may be requiredcoincident with an input picture (at 50 Hz) the second output picture5/6 of the way between the first 2 input picture, the third outputpicture 4/6 of the way between the 2nd and 3rd input picture and so on.This would give a sequence of temporal interpolation phases of 0, 5/6,4/6, 3/6, 2/6, 1/6, 0 and produce 6 output pictures for every 5 inputpictures. This is illustrated in FIG. 4. Note that the phase of eachtemporal interpolation always lies in the range 0 to 1. Strictly, thetemporal interpolation phase is the fractional part of the output timefor which an output picture is generated, expressed in input pictureperiods. The relative timing of input and output pictures is discussedin many texts dealing with digital sample rate changing, for examplereference 7.

For full motion compensation output pictures are generated for timeinstants corresponding to the temporal interpolation phase (seereference 3). This corresponds to the linear motion trajectory of FIG.1. For picture repeat output pictures are generated corresponding to thetime of the temporally nearest input picture, giving the motiontrajectory of FIG. 2. Intermediate motion trajectories can be achievedby generating output pictures corresponding to instants intermediatebetween the temporal interpolation phase and the time of the nearestinput picture. The extent to which the timing of output pictures ismoved from the temporal interpolation phase towards the nearest inputpicture time would depend on the reliability of the motion vectors fromthe motion estimator. By changing the interpolated motion trajectory ina continuous way a graceful fall back from full motion compensation topicture repeat can be achieved. This is the basis of the invention whichcan thereby achieve an acceptable interpolation method for all parts ofthe moving image even if the motion vectors are unreliable. Switchingartifacts, due to changing between interpolation modes, are avoided by acontinuum of motion trajectories between the two extremes.

A generic motion compensated interpolator is illustrated in FIG. 5. Theinterpolator has three inputs, a stream of input samples correspondingto the sequence of scanned input pictures, a stream of outputco-ordinates and a stream of motion vectors. The output co-ordinates arethe (spatio-temporal) co-ordinates for which values of the output imagesequence are calculated. They are generated by counters etc. asdescribed in the literature, for example references 3 & 6. The inputstream of motion vectors provides the motion vector associated with eachoutput co-ordinate. In each operating cycle a new output co-ordinate ispresented to the interpolator which (after a delay) generates the valueof the corresponding output pixel. The vector processor combines theoutput co-ordinates and corresponding motion vector to produce a set ofinput sample addresses and coefficient addresses for each outputco-ordinate (as described in reference 3). The output pixel value isgenerated by calculating a weighted sum of input pixel values. Thesample addresses correspond to the integer part of the required inputco-ordinate and are used to select the appropriate input pixel values,stored in the input store, and these are weighted by coefficientsselected from a precalculated set of filter coefficients stored in ROM.The filter coefficients are addressed by the fractional part of theinput co-ordinate calculated by the vector processor. The output valueis the sum of all the partial results presented by the set ofmultipliers. For brevity the diagram only shows two multipliers. Inpractice the number would probably be significantly more; 16 being atypical number for a motion compensated interpolator. Typically, theoutput pixel co-ordinate is measured in input fields and input picturelines. The motion speed is measured in input picture lines per fieldperiod. The size of the filter aperture is specified in terms of fieldsand lines, an aperture of 4 lines, therefore, corresponds to 8 picturelines. Because the input pixel values are addressed by the integer partof the input co-ordinate, the filter aperture is motion compensated tothe nearest integer number of field lines per field period. Theremaining, sub-pixel, motion compensation is achieved by varying thefilter coefficients. Further details of both non-motion compensated andmotion compensated interpolators can be found in the literature (e.g.references 3, 4, 6, 19, 21).

The motion compensated interpolator of FIG. 5 can be modified to provideadaptive motion trajectories controlled by the reliability of the motionvectors. This is illustrated in FIG. 6. Motion compensated interpolationis, in general, a 3 dimensional interpolation process. Consequently itshould be borne in mind that the output co-ordinates, presented to theinterpolator, comprise a 3 component vector. The components are thehorizontal, vertical and temporal parts of the output co-ordinates. Toproduce adaptive motion trajectories the temporal interpolation phase ispassed through a lookup table whose transfer characteristic iscontrolled by the reliability of the motion vector. The lookup tablecould conveniently be implemented using a Read Only Memory (ROM). Thetemporal interpolation phase is the fractional part of the temporaloutput co-ordinate; usually this is all that is presented to theinterpolator. In general for each output co-ordinate there can be adistinct corresponding motion vector and indication of vectorreliability associated with that motion vector. Hence the motiontrajectory can adapt on a pixel by pixel basis to obtain the bestinterpolation for each part of the image. Different parts of the imagecan, therefore, have different motion trajectories even if they have thesame motion vector because of the different levels of reliability of themotion vectors. Thus, better processing of regions having low vectorreliability can be achieved, for example, areas of revealed and obscuredbackground. These regions would be interpolated using temporally nearestpicture interpolation while other parts of the image might be fullymotion compensated.

The transfer characteristic of the look up table (LUT) in FIG. 6 iscontrolled by the vector reliability signal from the motion estimatorand determines the interpolated motion trajectory. Typical transfercharacteristics for the lookup table are illustrated in FIG. 7. Theoriginal temporal interpolation phase (φ_(in)) presented to the lookuptable is in the range 0 to 1. Assuming the reliability signal is alsoscaled to lie in the range 0 to 1 then a suitable transfercharacteristic for the lookup table would be given by equation 1.$\begin{matrix}{\varphi_{out} = {\frac{1}{2}\left( {1 - {\tanh \quad \left( \frac{{arctanh}\left( {{2\varphi_{in}} - 1} \right)}{r} \right)}} \right)}} & \text{Equation~~1}\end{matrix}$

where φ_(in) is the original temporal interpolation phase, r is thereliability of the motion vector and φ_(out) is the modified temporalinterpolation phase. Other sets of transfer functions for the look uptable are also possible.

The technique described above can be applied to motion compensatedtemporal interpolators described in the literature. The improvement isachieved by making allowance for the reliability of motion vectorsproduced, by an external motion estimation device, for the interpolator.The invention assumes the availability of a motion estimator whichprovides an indication of the reliability of the vectors it produces. Bytaking account of the reliability of the motion vectors objectionableswitching artifacts can be avoided, thereby improving picture quality.The invention allows the interpolation method used to change smoothlyfrom full motion compensation to non-motion compensation. This providesgraceful fall-back when violation of the assumptions underpinning motionestimation prevents the motion estimator measuring a reliable motionvector.

Graceful fall-back of motion compensated interpolation is achieved bymodifying the motion trajectory of moving objects in the interpolatedpictures. When the reliability of motion vectors is high a linear motiontrajectory is used corresponding to full motion compensation. Whenmotion vector reliability is low a stepwise motion trajectory is usedcorresponding to non-motion compensated interpolation. For intermediatevector reliability the motion trajectory used is intermediate betweenthese two extremes. Modulation of the motion trajectory is achieved bypassing the temporal interpolation phase, supplied to the interpolator,through a lookup table whose transfer characteristic is controlled bythe vector reliability.

This invention can also be used in conjunction with a motion detectorrather than a motion estimator. In reference 10 of the annex, a motionadaptive system is described in which interpolated images are producedusing temporal interpolation by applying a temporal filter aperturebetween successive fields. To avoid unacceptable artifacts such asdouble imaging when there is gross motion between successive fields, amotion detector is utilised to alter the temporal aperture on a pixel bypixel basis.

A motion adaptive system may be regarded as a motion compensation systemin which a single motion vector (zero) is used. The present inventioncan be implemented in a motion adaptive system, therefore, with themotion detector giving an indication of the reliability of the zeromotion vector. The invention is applicable in this case when the outputpicture rate is different to the input rate, for example, in standardsconversion or slow motion replay.

FIG. 8 implements the invention in a motion adaptive system. The outputco-ordinate processor in FIG. 8 is substantially the same as the vectorprocessor of FIGS. 5 and 6 except that there is no input for motionvectors as these are all notionally zero. The motion indication from themotion detector replaces the vector reliability indication in FIG. 6.

In this embodiment, as the motion across the aperture increases thereliability of the zero motion vector decrease and the phase of thetemporal interpolation is shifted towards the temporally nearest inputpicture. This system is an improvement over previous motion adaptivesystems as there is a reduction in multiple imaging and an improvementin the spatial resolution. Furthermore, the size of the coefficientstores can be reduced. If the temporal aperture is additionally variedwith the change in motion, then, larger coefficient stores are required.The use of motion adaptive systems provides a cheap and convenient wayof implementing the invention.

Whilst embodiments of the invention have been described, these are byway of example only and modifications will suggest themselves to thoseskilled in the art without departing from the scope of the invention asdefined by the appended claims. For example, the means of modifying thetemporal coordinate of the output coordinates may be other than by usinga look up table, for example, using suitable logic circuitry. Thisapproach provides an efficient implementation of a piecewise lineartransfer characteristic and could be embodied in field programmable gatearray or custom gate array integrated circuit. Modification of thetemporal coordinate may also be achieved using a state machine whichmight, additionally, use stored values of vector reliability(corresponding to spatially and temporally neighbouring pixels). In thislatter case, the modification of the temporal phase would depend on thevector reliability of neighbouring pixels as well as the current pixel.

REFERENCES

1. Aggarwal, J. K. & Nandhakumar, N. 1988 On the computation of motionfrom sequences of images—a review. Proc. IEEE, vol. 76, pp. 917-935,August 1988.

2. Bierling, M., Thoma, R. 1986. Motion compensating field interpolationusing a hierarchically structured displacement estimator. SignalProcessing, Volume 11, No. 4, December 1986, pp. 387-404. ElsevierScience publishers.

3. Borer, T. J., 1992. Television Standards Conversion. Ph.D. Thesis,Dept. Electronic & Electrical Engineering, University of Surrey,Guildford, Surrey, GU2 5XH, UK. October 1992.

4. Borer, T. J., Hulyer, M. G. & Parker, D. W. 1990. Motion compensateddisplay field rate upconversion. International Broadcasting Convention,Brighton, England, September 1990. IEE Publication No. 327. ISBN 085296000.

5. Chiariglione, L., Corgnier, L. & Guglielmo, M. 1986. Pre- andpost-processing in a video terminal using motion vectors. IBC. Brighton1986.

6. Clarke C. K. P. 1990. Interpolation. Digital Television edited bySandbank, C. P., John Wiley & Sons Ltd, 1990 ISBN 0-471-92360-5, pp287-374.

7. Crochiere, R. E. & Rabiner, L. R. 1983 Multirate digital signalprocessing. Prentice Hall, ISBN 0-13-605162-6.

8. Dubois, E., Konrad, J., 1990. Review of techniques for motionestimation and motion compensation. Forth international colloquium onadvanced television systems, Ottawa, Canada, June 1990.

9. Drewery, J. O., 1995. TV—The grand illusion. I.E.E. Colloquium onmotion reproduction in television. I.E.E. Digest No: 1995/093, 3rd May1995.

10. Ellis, G. 1995. Motion adaption—the everyday solution to a varispeeddisk playback. I.E.E. Colloquium on motion reproduction in television.I.E.E Digest No: 1995/093, May 3, 1995.

11. Girod, B. & Thoma, R 1985. Motion-compensating field interpolationfrom interlaced and non-interlaced grids. 2nd international technicalsymposium on optical and electro-optical science and engineering: Imageprocessing symposium, Cannes, December 1985.

12. Huahge, T. S., Tsai, R. Y., 1981. Image sequence analysis: Motionestimation. Image sequence analysis, T. S. Huange (editor),Springer-Verlag, Berlin, Germany, 1981, pp. 1-18.

13. Marcozzi, E. & Tubaro, S. 1987. Motion compensated interpolation.SPIE, Vol. 804, Advances in image processing, pp 61-68.

14. Martinez, D. M. 1987. Model-based motion estimation and itsapplication to restoration and interpolation of motion pictures. RLETechnical Report No. 530. June 1987. Research Laboratory of Electronics,Massachusetts Institute of Technology, Cambridge, Mass. 02139 USA.

15. Netravali, A. N., Robbins, J. D. 1979. Motion compensated televisioncoding, Part 1. Bell Syst. Tech. J., vol. 58, pp 631-670, March 1979.

16. Reuter, T. 1989. Standards conversion using motion compensation.Signal processing 16, 1989, pp 73-82.

17. Robert, P. Lamnabhi, M., Lhuillier, J. J. 1989. Advanced highdefinition 50 to 60 Hz standards conversion. SMPTE Journal, June 1989 pp420-424.

18. Thoma, R. & Bierling, M. 1989. Motion compensated interpolationconsidering covered and uncovered background.

19. Thomas, G. A., & Lau, H. Y. K. 1990. Generation of high qualityslow-motion replay using motion compensation. International BroadcastingConvention, Brighton, England. IEE Publication No. 327 ISBN 0 85296000.

20. Thomas, G. A., 1987 Television motion estimation for DATV and otherapplications. BBC Research Department report No. 1987/11. September1987.

21. Thomas, G. A., 1988. Distorting the time axis: Motion compensatedimage processing in the studio. International Broadcasting Convention,Brighton. England. September 1988. Published by IEE ISBN 0 85296368 8.

22. Thomson, R. 1995. Problems of Estimation and Measurement of Motionin Television. I.E.E. Colloquium on motion reproduction in television.I.E.E Digest No: 1995/093, May 3, 1995.

23. Tonge, G. J. 1986. Time sampled motion portrayal. Proceedings of thesecond international conference on image processing and itsapplications, Imperial College of Science and Technology, London Jun.24-26, 1986, organised by the I.E.E., pp 216-219.

24. Vega-riveros, J. F. Jabbour, K. 1986. Review of motion analysistechniques. IEE Proceedings, Vol. 136, Pt I., No. 6, December 1989.

25. Watkinson J. 1994. The Art of Digital Video 2nd Edition pp 238-264.Focal Press. ISBN 0 240 51369.13.

What is claimed is:
 1. A method of interpolating in processing of videoor film signals comprising: storing input pixel values of an inputsignal in an input store, assigning a motion vector to each set ofoutput coordinates to be interpolated, providing an indication of adegree of reliability of each said assigned motion vector, wherein thedegree of reliability has a predetermined upper and lower limit, andwherein the degree of reliability may be assigned at least oneintermediate value between the upper and lower limits, modifying atemporal coordinate of each said set of output coordinates depending onthe degree of reliability of the corresponding assigned motion vector,and selecting at least one pixel value from the input store depending onthe assigned motion vector and the modified temporal output coordinate,an interpolated output pixel value being determined from said at leastone pixel value; wherein each said temporal coordinate is modified usinga lookup table, a transfer characteristic of which is determined by thedegree of reliability of the corresponding motion vector; and whereinthe transfer characteristic used to modify each said temporal coordinateis given by the following equation:$\varphi_{out} = {\frac{1}{2}{\left( {1 - {\tanh \quad \left( \frac{{arctanh}\left( {{2\varphi_{in}} - 1} \right)}{r} \right)}} \right).}}$


2. A method of interpolating in processing of video or film signals asclaimed in claim 1, wherein the input store stores a plurality ofpictures, at least one pixel value is selected from each picture and acorresponding coefficient is selected from a coefficient store for eachpixel value, the value of the output pixel is determined from a weightedsum of the plurality of pixel values multiplied by their correspondingcoefficients.
 3. A method of interpolating in processing of video orfilm signals for motion compensated interpolation as claimed in claim 1,wherein the motion vector assigned to each set of output coordinates isassumed to be zero and a motion detector is used to give an indicationof how reliable a zero motion vector is for each set of outputcoordinates.
 4. A video or film signal processing interpolationapparatus comprising: an input store for storing pixel values of aninput signal, means adapted to assign a motion vector to each set ofoutput coordinates to be interpolated, means providing an indication ofa degree of reliability of each said assigned motion vector, wherein thedegree of reliability has a predetermined upper and lower limit, andwherein the degree of reliability may be assigned at least oneintermediate value between the upper and lower limits, means adapted tomodifying a temporal coordinate of each said set of output coordinatesdepending on the degree of reliability of the corresponding assignedmotion vector, and a vector processor which selects at least one pixelvalue from the input store depending on the modified temporal outputcoordinate and the corresponding motion vector; wherein said meansadapted to modify the temporal coordinate of each said set of outputcoordinates includes a lookup table; and wherein the lookup table has atransfer characteristic given by the following equation:$\varphi_{out} = {\frac{1}{2}\left( {1 - {\tanh \quad \left( \frac{{arctanh}\left( {{2\varphi_{in}} - 1} \right)}{r} \right)}} \right)}$

where Φin is the original temporal interpolation phase; r is thereliability of the motion vector; and Φ_(out) is the modified temporalinterpolation phase.
 5. A video or film signal processing interpolationapparatus as claimed in claim 4 and further comprising a plurality ofmultipliers and associated coefficient stores, and an adder, the inputstore being adapted to store a plurality of pictures, the vectorprocessor selects at least one pixel value from each of said pluralityof pictures and a corresponding coefficient for each said pixel value,the output pixel value being calculated from a sum of the selected pixelvalues weighted by the corresponding coefficient.
 6. A video or filmsignal processing interpolation apparatus as claimed in claim 4, andfurther comprising a motion detector, wherein the motion vector assignedto each said set of output coordinate is assumed to be zero and themotion detector supplies the indication of the degree of reliability ofa zero motion vector to the means for modifying the temporal coordinateof each said set of output coordinates.
 7. A video or film signalprocessing interpolation apparatus as claimed in claim 4, furthercomprising a motion estimation device which assigns motion vectors toeach said set of output coordinates and is adapted to provide theindication of the degree of reliability of each motion vector to themeans for modifying the temporal coordinate of each set of outputcoordinates.
 8. A video or film processing interpolation apparatus asclaimed in claim 7, wherein the motion estimator is a block matchingtype and the indication of the degree of reliability is given by thematch error.
 9. A method of interpolating in process in of video or filmsignals comprising: storing input pixel values of an input signal in aninput store, assigning a motion vector to each set of output coordinatedto be interpolated, providing an indication of reliability of each saidassigned motion vector, modifying a temporal coordinate of each said setof output coordinates depending on the reliability of the correspondingassigned motion vector by using a lookup table having a transfercharacteristic determined by the reliability of the corresponding motionvector, and selecting at least one pixel value from the input storedepending on the assigned motion vector and the modified temporal outputcoordinate, an interpolated output pixel value being determined fromsaid at least one pixel value, wherein the transfer characteristic isbased on the following equation:$\varphi_{out} = {\frac{1}{2}\left( {1 - {\tanh \quad \left( \frac{{arctanh}\left( {{2\varphi_{in}} - 1} \right)}{r} \right)}} \right)}$

where Φ_(in) is the original temporal interpolation phase; r is thereliability of the motion vector; and Φ_(out) modified temporalinterpolation phase.
 10. A video or film signal processing interpolationapparatus comprising: an input store for storing pixel values of aninput signal, means adapted to assign a motion vector to each set ofoutput coordinates to be interpolated, means providing an indication ofreliability of said motion vector, means adapted to modifying a temporalcoordinate of each said set of output coordinates depending on thereliability of the corresponding assigned motion vector, and a vectorprocessor which selects at least one pixel value from the input storedepending on the modified temporal output coordinate and thecorresponding motion vector wherein said means adapted to modifying atemporal coordinate includes a lookup table having a transfercharacteristic given by the following equation:$\varphi_{out} = {\frac{1}{2}\left( {1 - {\tanh \quad \left( \frac{{arctanh}\left( {{2\varphi_{in}} - 1} \right)}{r} \right)}} \right)}$

where Φ_(in) is the original temporal interpolation phase; r is thereliability of the motion vector; and Φ_(out) is the modified temporalinterpolation phase.
 11. A method of interpolating in processing ofvideo or film signals comprising: storing input pixel values of an inputsignal in an input store, assigning a motion vector to each set ofoutput coordinates to be interpolated, providing an indication of adegree of reliability of each said assigned motion vector, modifying atemporal coordinate of each said set of output coordinates according toa transfer characteristic which is dependant on the degree ofreliability of the corresponding assigned motion vector, and selectingat least one pixel value from the input store depending on the assignedmotion vector and the modified temporal output coordinate, aninterpolated output pixel value being determined from said at least onepixel value, wherein the modifying step is performed based on only afractional part of the temporal coordinate.
 12. The method of claim 11,wherein if the degree of reliability is above an upper predeterminedvalue, the modifying step is performed to provide a linear motiontrajectory.
 13. The method of claim 12, wherein if the degree ofreliability is below a lower predetermined value, the modifying step isperformed to provide a stepwise motion trajectory.
 14. The method ofclaim 13, wherein if the degree of reliability is between the upper andlower predetermined values, the modifying step is performed to providean intermediate motion trajectory between said linear motion trajectoryand said stepwise motion trajectory.
 15. A method of interpolating inprocessing of video or film signals comprising: storing input pixelvalues of an input signal in an input store, assigning a motion vectorto each set of output coordinates to be interpolated, providing anindication of a degree of reliability of each said assigned motionvector, modifying a temporal coordinate of each said set of outputcoordinates according to a transfer characteristic which is dependant onthe degree of reliability of the corresponding assigned motion vector,and selecting at least one pixel value from the input store depending onthe assigned motion vector and the modified temporal output coordinate,and interpolated output pixel value being determined from said at leastone pixel value, wherein the degree of reliability has a predeterminedupper and lower limit, and wherein the degree of reliability may beassigned at least one intermediate value between the upper and lowerlimits.